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OFFICIAL TITLE: Input Apparatus 



Field of the Invention 

The present invention relates to an input apparatus for a processing 
system, such as a personal computer. In particular, the present invention 
relates to an apparatus having a surface for touch events, such as movement 
of a finger, that can be used as a pointing device. 

Introduction to the Invention 

The evolution of powerful low cost computing systems has been 
accompanied by a parallel evolution of the human-machine interface. The 
interface includes both hardware and software components: a general 
purpose computer is provided with a pointing device and a keyboard for text 
input, and a monitor for displaying information. The software operates with 
the pointing device to enable the user to identify a preferred item among 
several items displayed on the monitor. Once the preferred item has been 
identified by the user, further operations are made within this context. With a 
high resolution monitor, a detailed display can be established, including many 
hundreds or thousands of such items that the user can identify. As a result, 
pointing device operations have become frequent and unavoidable. 

Tracking the movement of a finger across a surface would seem to be 
a good alternative to common pointing devices. Known devices that achieve 
this are incorporated into laptop computers. Two types of such sensor are 
known, a resistive type and a capacitive type. Both of these suffer from the 



restriction that only one point of physical contact can be made with the 
surface. In a large sensor, the user must avoid touching the surface with the 
wrist or any part of the hand other than the moving fingertip. This would make 
use of large area versions of these sensors at least as uncomfortable as 
using an ordinary mouse. As a result, finger tracking is constrained to a 
relatively small area, resulting in considerable inconvenience. 

A capacitive sensor that permits multiple touch points is developed 
under the name Tingerworks", by Rngerworks Corporation, Delaware, USA. 
The sensor is described in International Patent Application WO 938149A1 A 
profile of touch proximity to a surface is generated by measuring capacitance 
at a large number of individually amplified sensors. The electronic circuitry is 
expensive, and complex software is required to differentiate between a 
moving fingertip and other parts of the hand. 

An acoustic touch panel for detecting finger taps and their location is 
-described in European Patent Application EP 0474232A2, Four microphones 
at the corners of a rectangular touch panel pick up the same tap signal at 
different times, according to how far across the panel the sound has to travel. 
These times are compared and triangulation is used to identify the tap 
location. However, the apparatus does not track continuous movement of a 
fingertip across the panel, and so could not be used as a pointing device. 

Another type of acoustic mode input device is described in German 
Patent DE 4143364A1. This takes the form of a tablet having a rippled 



surface. As a stylus is moved across the surface, a noise is generated when 
the stylus tip encounters a ripple. Four microphones in corners of the tablet 
pick up these sounds, and triangulation is used to obtain an estimate of the 
stylus position. More precise differential movement tracking can be obtained 
by analysing sidebands imposed upon a naturally resonant frequency mode 
of the stylus. A limitation with this system is that it requires a stylus, and it has 
the same functionality as the stylus-operated graphics tablets that are in 
widespread use. The graphics tablet does not replace a mouse, because it is 
necessary to pick up a stylus. 

Continuous tracking of finger position in an acoustic sensor is 
described in International Patent Application PCT/GBOO/04635, by the 
present applicant As a fingertip moves across a surface, natural friction 
results in the generation of wideband acoustic noise. This will occur with any 
smooth but unpolished surface. The distance from the fingertip to each of 
several microphones is identified by analysis based on an attenuation model 
for sound waves propagating through the surface. Triangulation with 
information from at least three microphones is then used to identify the 
location of the moving fingertip. However, the attenuation model describing 
the acoustic properties of the surface does not provide a high enough level of 
accuracy for a pointing device. Furthermore, the requirement for at least 
three microphones would prevent such a device from being connected 
directly to the audio input of a computer soundcard. Additional analogue to 



digital and interface circuits would be required. 

Summary of the Invention 

It is an aim of the present invention to provide an improved apparatus 
for tracking the movement of a fingertip against a surface. 

According to a first aspect of the invention there is provided an 
apparatus for inputting data into a processing system, comprising a fricative 
surface for receiving user finger movements made against that surface, and 
transducer means arranged to receive sounds arising from the movement 
of a user's fingertip across said fricative surface. The surface has a direct 
path and at least one reflection path for acoustic conduction of surface 
sound to said transducer means, so that interference between said paths 
occuring at said transducer means results in interference artefacts that 
characterise the location of a fingertip on said surface. 

According to a second aspect of the invention there is provided a 
method of inputting data into a processing system, in which signals relating 
to user input movements on a surface are analysed to generate input data 
for said processing system. The said signals are generated by transducer 
means in response to sound arising from a user input gesture being made 
upon said surface; surface sound is transmitted to the transducer means by 
a system of acoustic transmission paths, including a direct path and at least 
one reflection path, and the location of the sound on the surface is 



identified by processing a characteristic of said path system with 
characteristics of other path systems of known surface location. 

According to a third aspect of the invention there is provided a 
method of processing a signal to determine a location of a sound source on 
a surface, wherein said signal has a characteristic that is determined by 
acoustic reflections in said surface, comprising the steps of: (1) 
characterising said location by transforming said signal to produce 
transformed data that reveals reflection artefacts and (2) identifying a 
location for said sound source by processing said transformed data with a 
reflection model for said surface. 

Brief Description of the Drawings 

The invention will now be described by way of example only with 
reference to the accompanying drawings, in which: 

Figure 1 shows a computer system, including a touch tablet, and a 
computer; 

Figure 2 details operation of the touch tablet shown in Figure 1; 
Figure 3 shows a plan view of the touch tablet shown in Figure 1 ; 
Figure 4 shows a cross-sectional view of the construction of the touch 
tablet shown in Figure 1; 

Figure 5 shows several alternate touch tablet configurations; 

Figure 6 details an enhancement of the acoustic properties of the 



touch tablet shown in Figure 1; 

Figure 7 shows a circuit diagram of the touch tablet shown in Figure 1. 
including a stereo connection for a sound card; 

Figure 8 details key components of the computer shown in Figure 1, 
including a processor, a memory and a standard computer sound card 
suitable for connection to the circuit shown in Figure 7; 

Figure 9 details the contents of the memory shown in Figure 8, 
including installer instructions and a touch tablet driver comprising 
instructions and templates; 

Figure Iff summarises the invention and shows the main acoustic 
transmission paths from a sound source (such as a moving fingertip) to an 
asymmetrically located transducer, forming an acoustic transmission system 
unique to the location of the sound source on the touch tablet; 

Figures 11 to 13 detail different transmission systems for several 
locations of a sound source on the touch tablet shown in Figure 1 ; 

Figures 14 to 17 detail respective spectral characteristics of the 
transmission systems shown in Figures 10 to 13; 

Figure 18 summarises the effect of the installer instructions shown in 
Figure 9; 

Figure 19 summarises operations performed by the processor shown 
in Figure 8 in response to the touch tablet driver instructions shown in Figure 
9, including steps for searching for a sound location and a step of combining 
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left and right results; 

Figure 20 details the operations performed when searching for a 
sound location in Figure 19, including steps in which a location is identified; 

Figure 21 details the operations performed during the step of 
identifying a sound location shown in Figure 20, including correlation steps; 

Figure 22 details the templates shown in Figure 9; 

Figure 23 illustrates the correlation steps performed in Figure 21 ; 

Figure 24 defines the linear correlation equation used in Figure 21 ; 

Figure 25 defines the step of combining left and right results shown in 
Figure 19; 

Figure 26 illustrates the searching process summarised in Figure 20; 

Figure 27 shows an alternative embodiment of the invention, in the 
form of a touch tablet in a laptop computer; 

Figure 28 illustrates an aspect of the construction used in the laptop 
embodiment shown in Figure 27; 

Figure 29 shows a further alternative embodiment, in which the touch 
tablet contains a radio link to the computer shown in Figure 1, or to another 
processing system; 

Figure 30 details the circuitry of the touch tablet shown in Figure 29; 

Figure 31 illustrates another alternative embodiment, in the form of a 
spherical input device comprising two hemispheres; 

Figure 32 details microphone placement in one of the hemispheres 



shown in Figure 31; 

Figure 33 illustrates combination of two hemispheres of the type 
shown in Figure 32 to form the spherical device of Figure 31; 

Figure 34 shows an input device having parallel-connected 

transducers; and 

Figure 35 shows the circuit diagram for the input device shown in 

Figure 34. 

Detailed Description of The Preferred Embodiment 

A personal computer system is shown in Figure 1, comprising a 
computer 101, a touch tablet 102, a keyboard 103, and a monitor 104. A 
CDROM disk 105 is provided for the installation of new instructions. The 
computer system is operated by a combination of text input on the 
keyboard 103 and graphical and cursor movement input using the touch 
tablet 102. The monitor 104 displays a graphical user interface (GUI) that 
enables an operator of the system to enter text and navigate a wide variety 
of combined textual and graphical displays. 

In the computer system shown in Figure 1, the touch tablet 102 
operates as a pointing device, replacing a standard computer mouse. The 
touch tablet is operated by sliding movement or tapping of a fingertip 
directly on its surface. A sharp object such as a pencil can be used to enter 
handwriting into the system. The CDROM disk 105 includes instructions 
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and data necessary to establish a device driver in the computer 101, so 
enabling the touch tablet 102 to be used in place of a mouse. 

The touch tablet 102 shown in Figure 1 is detailed in Figure 2. The 
palm of user's hand 201 can rest gently upon the surface of the touch tablet 
102. When a fingertip is moved across the surface of the tablet 102, 
random noise is generated at the point of Interaction between the finger 
and the surface, as indicated at 202. The noise is generated by a random 
process due to friction, and this will occur on any smooth but unpolished 
surface: The noise that is generated is similar in character to natural 
random noise generated by the wind, and the noise generated by thermal 
activities within semiconductor components. A surface capable of 
generating noise in this way is referred to as a fricative surface. A broad 
range of frequencies is generated continuously by the process of friction, 
and this continuous presence of noise at the location of the fingertip is what 
makes it possible to track its location. 

A shorter sound is generated by tapping on the surface of the tablet 
102, and this is suitable for roughly identifying the position of touch events 
analogous to the click of a computer mouse. The touch tablet 102 is 
connected to the computer 101 by a cable 203. 

Known acoustic methods for detecting the location of a touch event 
rely on the presence of at least three microphones in order to perform 
triangulation in the two dimensions of the surface to locate the sound 
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source. The requirement of at least three microphones makes such 
apparatus expensive because of the need for at least three analogue to 
digital converters. This simple fact precludes operation of such a touch 
tablet with an ordinary computer without additional analogue to digital 
conversion and interface circuitry, as computers are generally supplied only 
with a stereo microphone input. Furthermore, although continuous fingertip 
tracking is described in PCT/GB00/04635 (by the present applicant), the 
analysis method which is described uses an attenuation model of the 
acoustic properties of the touched surface, which is not accurate enough 
for use in a pointing device. 

The touch tablet 102 is further detailed in plan view in Figure 3. 
Under the surface of the tablet 102 are two microphones 301 and 302. 
These are piezo-electric transducers to which sounds are conducted from 
the location of the friction noise 202. The microphones are located near two- 
comers of the tablet 1 02. 

A cross-sectional view of the touch tablet 102 is shown in Figure 4. A 
smooth fricative surface 401 is provided so that a finger may move 
smoothly across it and generate noise. The preferred surface is Formica? 
However, many surfaces are suitable, including fabric, paper and wood. 
The substrate 402 acts in combination with the surface 401 to define 
acoustic conduction in the tablet. Preferably the substrate is made from 
fibre-board, although wood with a specific grain direction may be used as 
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an alternative. Aluminium, dense plastics and nylon are also suitable 
materials for the substrate 402. Honeycomb construction of a symmetric or 
asymmetric form may be helpful in obtaining a substrate with the most 
effective acoustic properties. The piezo-electric microphones 301 and 302 
are bonded to the substrate 402. The piezo-electric elements are formed on 
thin brass discs, so that once bonded to the substrate they become 
acoustically part of the substrate. The microphones 301 and 302 may be 
embedded in any way into the surface 401 and or substrate 402, in order to 
most efficiently generate electrical signals from sounds propagating across 
the tablet 102. Beneath the substrate is a layer of felt 403, which provides 
acoustic insulation from the base 404, which is a thin sheet of plastic. 

For simplicity in the description of the acoustics properties of the 
touch tablet 102 and other embodiments, the substrate is considered a part 
of the surface, and the surface will be considered as including any physical 
parts of the tablet that significantly affect acoustic characteristics relevant to 
touch detection and location, except when explicitly stated otherwise. 

The location of the microphones 301 and 302 on the touch tablet 
102 is asymmetric. The tablet 102 provides a two dimensional area for 
touch detection. However, the microphones are not arranged symmetrically 
with respect to both dimensions. There is an axis of symmetry half way 
between the microphones that divides the tablet in two, but this is the only 
axis of symmetry. The arrangement of the microphones will be considered 
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as being asymmetric because there are fewer axes of symmetry (there is 
only one) than there are dimensions (there are two) in which the sound 
source is located. The Invention provides this asymmetry in a touch tablet 
so that two-dimensional location of a sound source can be achieved using 

only two microphones. 

Other possible asymmetric touch tablet configurations are shown in 
Figure 5. A rectangular touch tablet is shown at 501, having only one 
microphone. There are advantages to using two microphones. However, in 
theory only one is required, and in some embodiments this may be of 
practical benefit. Microphone placement on another rectangular touch tablet 
520 is more complex than that shown in Figure 3. In the preferred 
embodiment shown in Figure 3, microphones are positioned either side of a 
single axis of symmetry halfway between the microphones and dividing the 
tablet in two. This configuration reduces the amount of processing required 
when finding the sound source location. However, in some situations, 
additional accuracy may be obtained from an arrangement such as that 
shown at 520, making the extra processing requirement worthwhile. Other 
configurations of the invention are shown at 530, 540, 550 and 560. These 
are not intended to be exhaustive, and are merely illustrations of the 
requirement for microphone asymmetry. A large number of microphones is 
shown at 550, where the asymmetric condition is still satisfied, but more 
than two microphones is being used. This provides an extremely robust 
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touch location system which may be useful in a noisy environment. Any of 
these examples 510, 520, 530, 540, 550 and 560 could be plan views, of a 
three-dimensional shape, with the surface being curved in a third 
dimension. 

The asymmetry of the touch tablet 102 results in a high complexity of 
acoustic phenomena that are used to unambiguously identify the location of 
a noise source. Touch tablets designs such as 550 and 560 shown in 
Figure 5 are fairly complex. However, a regular shape for the touch tablet 
may be preferable, such as the rectangle of the preferred embodiment 102, 
but in which acoustic complexity needs to be increased to enable noise 
source location to be performed most accurately. The acoustic shape can 
be modified without changing the physical shape, and this is illustrated in 
Figure 6. In this alternative embodiment, the tablet 102 has a substrate 402 
made from wood having a grain in a particular direction. The direction of the 
grain is illustrated by vertical lines 601. The velocity of sound in wood is 
greater in the direction of its grain. When constructed in this way, the tablet 
102 has an acoustic width to height ratio 602 that is different from its 
physical width to height ratio; and this results in a beneficial increase in the 
complexity of acoustic artefacts which can be used to identify sound 
location in a rectangular touch tablet. Additional modifications are possible. 
For example, the thickness of the substrate 402 can be varied in a regular 
or irregular pattern or texture. 
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The circuit components for the touch tablet 102 are shown in Figure 
7. Each microphone is a piezo-electric transducer in the form of a disk. A 
thin brass disk 702 has a deposit of piezo-ceramic material 703 about 
0.5mm in thickness. The surface of the piezo-ceramic material is silvered, 
and electrical connections 704 are established with the silvered surface and 
the brass disk 702. In the circuit shown in Figure 7, each transducer 301 
and 302 has voltage limiting diodes 705, 706. 707 and 708, which prevent 
excessive voltages developing that could damage microphone input circuits 
in the computer 101. The transducer signals are connected by a screened 
audio cable 709 to a 3.5mm stereo plug 710. 

The computer 101 shown in Figure 1 is detailed in Figure 8. A 
PentiumTentral processing unit (CPU) 801 executes instructions held in 
one hundred and twenty-eight megabytes of main memory (RAM) 802, to 
perform manipulations of data also stored in the main memory 802. 
Instructions and data are permanently stored on a non-volatile hard-disk 
drive 803. Instructions for the processor may be installed from the CDROM 
105 inserted into a CDROM/DVD drive 804. A video card 805 receives 
graphical drawing commands from the processor 801 to render images on 
the screen of the monitor 104. 

A sound card 806 receives audio signals from the touch tablet 102 
through its microphone input socket, which accepts the plug 710 shown in 
Figure 7. The sound card 806 converts analogue electrical signals from its 
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microphone input into pairs of sixteen bit samples at a rate of 44100 left 
and right samples per second. This data stream is made available to 
applications running on the computer, by transfer between the sound card 
806, the processor 801 and the main memory 802. The sound card is also 
able to receive sound samples from applications and supply these to an 
amplifier, headphones or a small pair of loudspeakers that can be 
connected to the computer 101. Stereo sound cards of this type are built in 
to most general purpose computers. Often, the microphone input is of a low 
quality, with the analogue to digital conversion process having an effective 
accuracy of much less than sixteen bits. However, the touch tablet does not 
require significantly high quality analogue to digital conversion, 

A universal serial port 807 connects the keyboard 103 to the 
computer. A modem 808 facilitates connection to an Internet service 
provider (ISP) via an ordinary telephone line. Instructions and data may be 
installed from a site on the Internet, as an alternative to installation from the 
CDROM 105. 

Initialisation of the computer is performed when it is switched on, in 
accordance with bootstrap instructions for the processor 801. Once the 
bootstrap process is completed, the user typically starts and uses a number 
of software applications, such as a word processor, email and a web 
browser. These applications are often held in memory 802 at the same time 
so that the user may transfer data between them conveniently, and display 
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them in several overlapping windows on the monitor 104. 

The contents of the main memory 802 are shown in Figure 9. A 
Windows xp'operating system 901 provides a hardware-neutral application 
programming interface (API) for several applications running on the 
computer 101. The operating system includes installer instructions 902 for 
installing new applications or other types of instructions and data. Several 
device drivers 903 are provided to enable the operating system to co- 
operate with specific hardware devices in the computer system 101. For 
example, a device driver is provided for the video card 805 and for the 
sound card 806. A touch tablet device driver 904 is provided, and this 
includes touch tablet Instructions 905 and templates 906. Several 
applications 907, 908 and 909 are held in the memory 802, including their 
instructions and their data. The touch tablet driver 904 allows the 
applications to receive user commands from the touch tablet as if they were 
operations of an ordinary mouse or other pointing device. The many 
applications that are available for a computer 101 of this type can operate 
directly with the touch tablet 1 02 without modification. 

The main operations performed by the touch tablet driver 904 are the 
analysis of audio signals from the microphones 301 and 302. From this 
analysis, the same data that would ordinarily be generated by an ordinary 
pointing device can be generated by the touch tablet driver 904 in response 
to user finger movements and taps. 
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The analysis is based upon a reflection model of sound wave 
propagation in the touch tablet 102. The invention is explained in Figure 10, 
in which reflection paths of sound wave propagation are illustrated. The 
location of the fingertip moving across the surface of the tablet 102 is 
shown by a cross at 1001. Sound is transmitted outwards from this source 
1001 through the plane of the touch tablet 102. This results in a system of 
several acoustic transmission paths from the source 1001 to the 
microphone 301. 

The strongest transmission paths are shown. A single direct path 
1011 provides the shortest route from the source 1001 to the microphone 
301. Another path 1012 results from a single reflection 1013 against the 
near edge 1014 of the tablet 102. Another path 1015 results from a single 
reflection against the right edge 1016 of the tablet 102. A double reflection 
path 1018 results from a first reflection 1019 against the near edge 1014 
and a second reflection 1020 against the right edge. The relative strengths 
of the direct, single reflection and doubie reflection paths are indicated by 
the thickness of their lines. Each reflection results in an attenuation of the 
sound. Many other paths are present in the tablet, but are omitted for the 
sake of clarity. The combination of all such paths is considered as a 
system. For an asymmetric transducer location 301, there is a unique 
characterising system for every possible sound location 1001. 

Each path in the system shown in Figure 10 has a different 
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transmission time, so the original sound is combined in several different 
delayed versions as it is converted into electrical form by a transducer 301. . 
The magnitude of each delayed version is also different. Constructive and 
destructive interference occurs where the paths meet at the microphone. 
The interference pattern is different for each frequency in the spectrum. 
Since the sound is noise of a relatively continuous spectrum, the spectrum 
of sound received at the microphone 301 provides a clear impression of 
interference artefacts in the frequency domain. As the location of the sound 
source moves, its characteristic spectrum evolves also. Since the 
interference results from combination of a direct path 1011 with reflection 
paths, the spectrum may be considered as revealing reflection artefacts. 

The noise generated by a moving fingertip may be partially or 
entirely stochastic in origin, and has an overall smooth spectral shape, as is 
the case with 1/f noise. The overall shape of the amplitude probabilities 
through the spectrum is smooth. Constructive and destructive interference 
at the microphone 301 modifies the spectrum sharply, from a smooth into a 
complex shape. It is this complexity that enables the location of a sound 
source to be identified. 

• in the preferred embodiment, reflections are caused by the sound 
reflecting from edges of the tablet 102. However, it is possible for 
anomalies to be present in the tablet, so that reflections can occur from 
parts other than the edges, which may help to uniquely characterise the 
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location of a sound. Constructional anomalies may be used therefore, both 
to modify the velocity of sound in the tablet medium as shown at Figure 6, 
and also to introduce additional causes of reflection. Both these effects can 
be used in the construction of a tablet to ensure unambiguous mapping 
between a sound source location and artefacts arising from its system of 
reflections. 

The touch tablet 102 provides an apparatus for inputting data into a 
processing system. The surface of the tablet 102 permits movements of a 
user's fingertip to be formed upon it. These movements result in sounds 
1 001 being generated, and transducer means, in the form of microphones 
301 and 302, picks up sounds arising from these movements. A pencil may 
be used in place of the user's fingertip, and a sheet of paper may be placed 
over the tablet's surface so that handwriting can be entered as data into the 
processing system. Acoustic transmission of the surface sound 1001 to 
transducer means 301 and or 302 is characterised by a system of paths 
1011, 1012, 1015, 1018, including reflections 1013, 1016, 1019. 1020. The 

asymmetry of the transducer's location 301 and or 302, with respect to 

r 

acoustic properties of the surface, has the resulting effect that sound 
source locations on the surface, such as location 1001, are uniquely 
characterised by their system of transmission paths 1011, 1012, 1015, 
1018. 

In Figure 11 a system for a different location 1101 is illustrated. The 
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direct transmission path 1121 is shorter, and the other main transmission 
paths also have different lengths and therefore transmission times and 
relative amplitudes. They are added together at the microphone 301 in 
accordance with their different amplitudes and delay times. Comparison 
between Figures 10 and 11 shows that the transmission systems differ 
significantly according to the location of the sound. The difference is 
complex, but varies smoothly as the fingertip moves continuously across 
the surface from location 1001 to location 1101. Systems for other sound 
source locations are illustrated in Figures 12 and 13. 

Each location on the surface of the tablet 102 has its own unique 
system of sound propagation paths, which, through constructive and 
destructive interference at the transducer, can be revealed by transforming 
the transducer signals into location characterising data that reveals 
reflection artefacts. These artefacts exist because reflection paths combine 
with each other, and with the direct path, to form system characteristics 
which can be revealed by a transformation upon samples representing the 
sound received at one or several microphones. 

The spectrum of frequency magnitudes is obtained by fourier 
transformation of the microphone. The resulting magnitudes of frequency 
components are indicative of the system, and hence the location of the 
sound source. These magnitudes are considered as a form of location 
chacterising data. The spectrum of a moving sound source at the location 
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shown in Figure 10 is shown in Figure 14. Corresponding spectral plots for 
the remaining three systems shown in Figures 11, 12 and 13, are shown in 
Figures 15, 16 and 17 respectively. These may be referred to for 
convenience as spectrograms. Each spectrogram is complex in shape, and 
uniquely characterises the transmission system for a sound source at a 
particular location on the tablet. Other forms of location characterising 
transformation may be possible, and the invention is not restricted to the 
use of the fourier transformation, or similar transformations. 

It is possible to identify a sound source location in two dimensions 
uniquely from the reflection artefacts inherent in the signal from a single 
microphone 301. This reduction in the number of microphones compared to 
the prior art makes it possible to connect an acoustic touch tablet to the 
stereo input of an ordinary computer 101 without an interface circuit. 
Furthermore, the uniqueness of the mapping from reflection artefacts to 
sound location works at a high level of detail, so facilitating a level of 
resolution suitable for use in a pointing device, and also for handwriting 
input. 

In the preferred embodiment, two microphones 301 and 302 are 
used. This has advantages for the rejection of false signals due to ambient 
noise. Furthermore, noise generated by a large area, for example the palm 
of the hand, will not be identified as having a specific location by the 
present system. 
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For the microphone arrangement shown in Figures 10 to 13, each 
microphone will see a different system of paths from a particular sound 
location 1001, except when the sound is located exactly halfway between 
the two microphones. This does not result in any ambiguity, since the 
location is not determined by a process of triangulation as in the prior art. 
The two different microphone signals are used for confirmation and 
subsequent refinement of a location. The set of possible systems for the left 
microphone 301 is a mirror image of the set of possible systems for the 
right microphone 302. This relationship can be used to gain efficiency in 
computation and a reduction in the number of templates 906 that is 
required. 

In some embodiments, such as touch tablet 540 shown in Figure 5, 
two possible locations may be characterised by the same system of paths. 
In this type of microphone arrangement, the combination of microphones is 
considered as providing the asymmetry required to unambiguously locate a 
sound. In this respect, therefore, the transducer means for the touch tablet 
is still considered as being asymmetrically located. 

Instructions for the installer 902 may be run by the user in order to 
load the touch tablet driver 904 onto the computer 101. The effect of the 
installer instructions 902 is summarised in the flowchart shown in Figure 18. 
At step 1801 a question is asked as to whether the touch tablet driver 904 
has already been installed. If so, no installation steps are necessary. 
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Alternatively control is directed to step 1802 where a question is asked as 
to whether the driver 904 is to be loaded from a site on the Internet. If not, 
control is directed to step 1804. Alternatively, at step 1803, the user 
connects to the Internet and downloads the installation package for the 
touch tablet driver 904. Thereafter control is directed to step 1805. If the 
touch tablet driver 904 is not to be downloaded from the Internet, control is 
directed to step 1804. The user places the CDROM disk 105 in the 
CDROM/DVD drive 804, and locates the file containing the installation 
package for the touch tablet driver 904. Thereafter control is directed to 
step 1805, where the touch tablet driver is installed, by a process of data 
decompression and storage on the hard disk 803, along with an update of 
associated files used by the operating system 901 to indicate the presence 
of the driver on the system and its characteristics. In the Windows XP 
operating system, it is necessary to restart to computer system after 
installation of a device driver. 

Operation of the instructions 905 of the touch tablet driver 905 are 
summarised in the flowchart shown in Figure 19. The steps of Figure 19 are 
operated in a continuous loop as a process among many other 
simultaneously executing processes that are established by the operating 
system 901. Touch events are of three different types. A transient tap, an 
initiation of a movement and a continuation of a movement. The driver waits 
for the start of a new touch event, which may be a tap event, or the start of 



25 



a finger movement. At step 1901 a question is asked as to whether any 
significant level of audio signal is being received by the microphone input of 
the sound card 806. If not, control is directed back to step 1901. and this 
sequence repeats until a suitable signal level is detected in the audio 
samples. 

A fourier transformation is performed on a set of audio samples at 
step 1902. Samples received from the sound card 806 are received in 
packets of two hundred and fifty-six samples each for the left and the right 
microphones. A banning window is applied to each set of two hundred and 
fifty-six samples, and then a fast fourier transform (FFT) is used to 
efficiently compute a frequency domain representation of the left and right 
signals. The frequency domain representation also has two hundred and 
fifty-six data items per channel, each representing the magnitude of a 
different frequency in the audio spectrum. The sample rate used by the 
sound card is 44.1 kHz, and the fourier transform is used to provide an 
indication of frequency component magnitudes up to 22.05 kHz. In an 
alternate embodiment, it is possible that another type of transformation may 
be used, for example a wavelet transformation. The purpose of step 1 902 is 
to transform the audio samples from the sound card in a way that reveals 
location information about the sound source 202 in the form of reflection 
artefacts. When the magnitudes of frequency components are known, this 
reveals information about the system of acoustic transmission paths that is 
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unique to each possible sound location. 

At step 1903 a search is performed for an initial location by 
processing the fourier transform of the left and then the right channel with 
templates 906. At step 1904 a question is asked as to whether the search 
for an initial location generated a valid location. If the answer is no, control 
is directed back to step 1901. Alternatively, control is directed to step 1905, 
where the overall amplitude profile of the original time domain audio 
samples is examined to determine a sharp or a smooth initial amplitude 
transition within the two hundred and fifty-six samples available. A sharp 
transition indicates a finger tap event, equivalent to a . mouse click, and a 
smooth transition indicates a movement event. At step 1906 a question is 
asked as to whether a tap or a move event has been detected, and control 
is directed to step 1 91 1 or step 1 907 accordingly. 

At step 1907 movement data is transmitted to the operating system 
901 so that pointer movement information can then be supplied to 
applications 907, 908 and 909. At step 1908 another pair of left and right 
fourier transforms is obtained, in the manner described for step 1902. A 
search for a nearby location is made at step 1909. During the continuous 
movement of a fingertip across the surface, successive positions will be 
located near each other. This permits a much reduced set of templates 906 
to be used for processing at step 1909, thereby reducing the amount of 
calculation necessary to track continuous movement of a fingertip. At step 
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1910 a question is asked as to whether the search for a nearby location 
has been successful. If so, control is directed to step 1907, where 
movement data is again transmitted to the operating system 901. The loop 
comprising steps 1907 to 1910 is repeated until valid incremental 
movements are no longer detected. After this, control is directed to step 
1901, where a search is made for another initial position of a touch event. 
Tap information is transmitted to the operating system 901 at step 1911. 
whereafter control is directed back to step 1901. 

Analysis to obtain the location of a touch event is performed similarly 
at both steps 1903 and 1909 in Figure 19. Both these are detailed in the 
flowchart shown in Figure 20. At step 2001 a rough sound location is 
identified, and a separate rough location is Identified for each of the left and 
right channels. At step 2002 a question is asked as to whether locations 
indicated by left and right channels are close to each other. If not, it is 
presumed that the audio signal does not come from a point source of 
sound, and probably results from an ambient sound picked up by the tablet 
102 or other unintentional source. Steps 2001 and 2002 are performed only 
when the flowchart of Figure 20 applies to step 1903, where an initial 
location is required. This is because, in the absence of any prior knowledge 
of the location of the sound source 202, it is necessary to consider all 
possible locations on the tablet 102. It would require a lot of processing to 
identify the sound location to a high level of accuracy in a single attempt, so 
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a rough location is obtained as a preliminary step. The remaining steps 
2003 to 2007 are performed after an initial rough location is known. The 
rough location may be provided by steps 2001 and 2002, or from 
knowledge of the previous location occurring as part of a continuous finger 
movement, during which the previous position provides a good estimate of 
the current position. Methods such as position history-based dead 
reckoning (PHBDR) can be used to improve rough location estimation 
during continuous movement, and thereby reduce the amount of 
processing required still further. 

Detailed searching of the local area around the estimated location is 
performed at step 2003. The same fourier transformations of the left and 
right channels are used in comparison with a selected subset of templates 
associated with the locality of the estimated location. As a result of step 
2003, several template scores are generated, for each of the left and right 
channels. The template scores are combined with locations associated with 
each template by a process of interpolation performed at step 2004. This 
generates a more accurate pair of locations of the sound source. 

At step 2005 a question is asked as to whether the left and right 
positions agree. If they are significantly different, the locations are 
discarded. Alternatively they are combined in proportion to their respective 
overall correlation scores to generate a single pair of x and y co-ordinates 
for the finger tap or move position. At step 2007 a flag is set in order to 
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indicate to step 1904 or step 1910 that the location of the sound has 
successfully been identified. 

Identifying a sound location is achieved by comparing the left and 
right fourier transformations with a set of templates 906. This process, 
shown at steps 2001 and 2003 in Figure 20, is detailed in Figure 21. At step 
2101 a first template is selected for comparison. At step 2102, a process of 
linear correlation Is used to obtain a score of similarity between the shape 
of the spectrum from the left microphone, and the shape of the spectrum 
stored by the template. Each template corresponds to a known location on 
the tablet's surface, and scores generated in this way for templates 
corresponding to several positions can be compared to provide an 
indication of the location of the sound source 202. 

At step 2103 the same template is correlated with the right channel 
spectrum, and a score Indicative of the similarity is again generated. At step 
2104 a question is asked as to whether another template is to be 
considered. If so, control is directed back to step 2101. These steps are 
repeated until all appropriate templates have been considered. When 
obtaining a rough location at step 2001, only templates corresponding to a 
sparse set of locations over the whole tablet are considered. When 
obtaining an accurate location at step 2003, only templates corresponding 
to locations in a local area around and including the rough location are 
correlated. 



After template scores for the rough location have been obtained, at 
step 2105 a square of nine areas is identified. These correspond to the 
highest scoring square of nine templates that have been considered for the 
left channel. At step 2106 the same process is repeated for the right 
channel template scores. 

To summarise: when obtaining a rough location, a first area is 
identified on the tablet. This area comprises locations associated with the 
best-scoring templates. Thereafter, when refining the search to a higher 
level of accuracy, a subsection of the large area is identified. The 
subsection area includes a central set of nine locations, Interpolation can 
then be performed, at step 2004, to determine the actual location of the 
sound source to a higher level of accuracy than that represented by the 
locations associated with the templates 906. Separate results for left and 
right are compared and combined, at steps 2005 and 2006, in order to 
identify the validity of the results, and to further increase their accuracy if 
they are validated. 

The templates 906 shown in Figure 9 are detailed in Figure 22. The 
templates are data records 2201, of which two thousand three hundred and 
thirty-four are required for the entire template data set 906. The templates 
906, along with instructions 905, form a reflection model of the touch tablet 
102. An individual template is detailed at 2202. Each template includes a 
spectrogram 2203 representing the normalised spectrum of frequency 
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magnitudes of a simulated noise source at a particular location on the tablet 
surface. The spectrogram represents the magnitudes of each of two 
hundred and fifty six frequency components generated by the same 
mathematical transformation that is used at step 1902 in Figure 19. The 
location of the simulated noise source is stored at 2204 and 2205. Values 
2204 and 2205 provide x and y co-ordinate values for left microphone 301 
signals, whereas values 2206 and 2207 for the right microphone 302 are 
reflected in the x dimension, exploiting the fact that the locations of 
microphones 301 and 302 are reflections of each other about a single 
central axis. Value 2206 is equal to the width of the touch tablet 102 minus 
the corresponding value 2204 for the left channel. This arrangement cuts in 
half the number of templates that would otherwise be required. 

The process of correlation shown at steps 2102 and 2103 in Figure 
21 is detailed in Figure 23. The spectrum of frequency magnitudes 
generated for a microphone channel comprises two hundred and fifty six 
different frequency magnitudes, and is shown at 2301. This set of two 
hundred and fifty six data values is compared with the corresponding set of 
two hundred and fifty-six values in each template 2302 to 2306, to generate 
a respective score 2307 to 2311. The comparison is a process of linear 
correlation, resulting in a potential score range from minus one to plus one, 
reflecting the level of similarity between the shape of the microphone 
spectrum and the template against which it is being compared. 



The linear correlation process used to compare a microphone 
spectrogram with a template is shown in the form of an equation in Figure 
24. This is the standard linear correlation equation, in which the average of 
each data set is subtracted from each of its data items, and the product of 
paired values generated in this way is accumulated and then normalised to 
a score within the range minus to plus one. The microphone spectrum data 
set is indicated by the letter a, with subscript letter i being used to indicate 
selection of a particular frequency in a summation process. The template 
values are indicated by the letter n. This calculation results in the 
correlation coefficient, r, being generated, which is a measure of the 
similarity in shape of the two data sets that are being compared. 

A value for r is generated for left and right channels separately when 
a template comparison is performed. In Figure 25, these are denoted r with 
a subscript L or R. Each template also has an associated x and y co- 
ordinate pair for each of left and right channels. The equation in Figure 25 
combines left channel and right channel results to generate a single overall 
x and y co-ordinate pair to a high level of accuracy, detailing the process 
performed at step 2006. 

The process of identifying a rough location and then refining it is 
illustrated in Figure 26. The tablet 102 is initially considered as comprising 
an overall area of sixteen by sixteen small rectangular areas 2601, 2602, at 
the centre of which is a point whose characteristic spectrum is recorded in 
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one of the templates 906. The initial rough search performed at step 2001 
in Figure 20 considers these two hundred and fifty-six possible locations, 
finally identifying a rough location in the form of a three-by-three high- 
scoring region 2603 which has the highest set of correlation scores of any 
possible three-by-three area on this scale. An initial high scoring region 
2603 is generated for each of left and right microphone channels 
separately. If these areas overlap, this is considered as indicating that a 
valid location can be identified from the sound source. 

The initial area 2603 is then considered at a higher level of template 
resolution, in which each of the initial area subdivisions 2601, 2602 is 
considered as being further subdivided into smaller areas 2604, 2605. At 
the centre of each of these smaller areas is a point whose spectrum is 
recorded by one of the templates 906. The high scoring region 2603 is 
further examined by considering templates from the centre outwards, until it 
becomes clear that a peak region of three-by-three small areas has been 
identified. The peak region is shown at 2607. The correlation scores are 
shown for each of the small areas that has had to be considered in order to 
identify the peak region 2607. These values are processed (at step 2004) 
using bicubic interpolation. This identifies a finely resolved location 2608 to 
a higher accuracy than that provided by the raw x and y co-ordinates 
associated with each template. Two such locations are generated, one for 
each of left and right. These are combined at step 2006. assuming that they 
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are close enough together to be considered as producing a valid location. 

In summary, the system of template analysis enables locations of a 
moving fingertip or finger tap to be identified. The system of templates 906 
stores characteristics of a particular tablet design. The set of data defined 
by the templates, and the way in which they are used, may be considered 
as providing a reflection model. Preferably, template analysis is used. 
However, other techniques may be successfully applied to providing a 
reflection model. An alternative approach is neural network processing. 
This may be of particular benefit when a purpose-built neural network 
processor is used, thereby making it possible to implement the large 
number of artificial neurones that would be required for reflection model of 
the touch tablet 102. 

Repeated analysis can track the movement of a continuously-made 
stroke of a fingertip or a pencil. The resulting movement data may be used 
in a conventional way, so that the tablet may be used as a pointing device 
to replace a mouse. As an alternative, movement of a finger can draw 
characters in a pictographic alphabet, which is of particular benefit for those 
alphabets that have too many characters for a conventional computer 
keyboard 103. The touch tablet may be seen as a way of inputting user 
data to a processing system, whether that be in the form of pointing actions 
to control a cursor on a screen, or in the form of direct data input, such as 
handwriting or input of pictographic characters. 
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In an alternative embodiment, the invention is used in a laptop 
computer, which is shown in Figure 27. The entire area 2701 in front of the 
keyboard 2702 can be used as a touch tablet, with the advantage that 
much longer continuous finger movements can be accommodated 
compared with the small area that is traditionally made available. There are 
several reasons why known laptop touch pads are so small. These include 
accuracy and cost. However, a major problem is that laptop users use the 
large area in front of the keyboard to rest their wrists while typing, and this 
would result in false signalling if a known sensor were to include this area. 
In the invention, only concentrated localised sources of sound are 
recognised as having distinct locations, and so it is possible to use the 
touch tablet area 2701 as support during typing without the usual problems 
occurring. In a further variation, the keyboard 2702 can be removed 
entirely, with the touch tablet 2701 extending over the entire area opposite 
the display. Text and alphanumeric input may be achieved by symbolic 
gesture input. A similar scheme might be employed in a smaller device, 
such as a personal digital assistant (PDA). 

Cross-sectional construction details for reducing false signalling in 
the laptop touch tablet 2701 are shown in Figure 28. A laptop computer has 
several sources of internal noise, including a processor fan and internal 
loudspeakers. An insulating felt layer 2801 acoustically isolates the tablet 
2701 from the chassis 2802 of the computer 2700. The underside of the 
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tablet area may be acoustically insulated using acoustic wadding. The 
internal loudspeakers are located in the base of the display portion of the 
case, to reduce interference when sounds are emitted. 

In another alternative embodiment, the transmission of microphone 
signals from the touch tablet to the computer 101 is performed over a radio 
link, as indicated in Figure 29. Circuitry for the radio touch tablet 2901 is 
detailed in Figure 30. Analogue to digital converters (A-D) 2902 supply 
digitised microphone signals to a Bluetooth baseband and radio frequency 
circuit 2903. The stream of digital audio from the A-D converters 2902 is 
compressed by a digital signal processor (DSP) in the Bluetooth*c!rcuit 
2903. The compression is a process of near instantaneous compression 
(NICAM), and is necessary in order to meet the relatively low bandwidth 
requirements of a Bluetooth radio channel. 

Radio signals are transmitted via an antenna 2904. A rechargeable 
power source 2905 includes a NiMH cell, regulator and charging circuitry, 
which is connected to an external charger through a charging socket 2906. 
The rechargeable power source 2905 also includes a visual indicator (LED) 
on the side of the tablet, to indicate charging status during use, and during 
recharging. The microphones 2907 and 2908 used in the radio touch tablet 
2901 are the same as those described in the preferred embodiment. 
Provision of the Bluetootrfradio link enables the tablet 2901 to be used with 
diverse processing systems, including mobile phones, personal digital 
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assistants (PDAs) and laptop computers. The radio link has extremely low 
power consumption, with most of the power for the radio tablet being 
consumed by the analogue to digital converter 2902. 

In PCT/GB00/04635. by the present applicant, symmetrical 
microphone positions are used to obtain the location of a touch event upon 
the surface of a fully enclosed sphere. Distance from the sound source to a 
microphone is identified with reference to an attenuation model for sound 
propagation. The method described uses the acoustical properties of a 
spherical surface in which reflections do not occur. Templates are used, but 
the simplicity of the attenuation model requires only four templates, making 
the disclosed method several orders of magnitude simpler than is required 
by a reflection model. A spherical sensor of this type is difficult to 
manufacture, and the attenuation model would not provide the level of 
resolution required for a pointing device, or the precise navigation of 3D 
space, which is another of the uses of a spherical sensor of this type. 

An improved spherical sensor is illustrated in Figure 31. The sensor 
3101 rests in the one hand and is operated by a finger of the other hand. 
The sensor 3101 can be freely rotated and manipulated. The sensor 3101 
transmits touch and orientation information over a radio link. 

A hemisphere is shown in plan view in Figure 32. The hemisphere 
3200 contains two microphones 3201 and 3202 that are located 
asymmetrically under its surface. Acoustic reflections from a flanged lip 
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3203 of the hemisphere ensure asymmetrical acoustic properties for the 
microphones when they are considered together. This can be achieved by 
electrically combining signals at their output, or by performing template 
analysis separately for each microphone signal. Each channel generates 
two possible locations, and these are resolved by combination. As a further 
alternative, full acoustic asymmetry for each individual microphone can be 
achieved by constructing the hemisphere of an irregular varying thickness 
of material. As yet another possibility, many microphones can be combined 
electrically in several combinations, resulting in two channels of differently 
mixed microphone signals. This achieves a rich complexity of acoustic 
systems unique to each possible location of the sound source on the 
surface of the hemisphere 3200. 

In Figure 33, two such hemispheres 3200 and 3300 are connected 
together to form the sphere 3101 shown in Figure 31. Although physically 
connected, they remain acoustically independent, with the required 
reflection processes occurring from the flanged edge 3201 of each 
hemisphere 3200 3300 where the two hemispheres meet at the equator of 
the ball. Such construction provides the acoustic qualities of a reflection 
model, suitable for high precision touch event location. Replacement of 
components, such as a rechargeable battery, can be performed by simply 
unscrewing the two halves. 

Combining multiple microphones into a single channel is illustrated in 
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Figure 34. A hemispherical or flat circular sensor 3401 has six piezo-electric 
microphones 3411, 3412, 3413, 3421, 3422 and 3423. Three microphones 
341 1 to 3413 are combined for a first channel, connected to the left channel 
of the stereo input to the computer sound card. The remaining three 
microphones 3421 to 3423 are combined for a second channel, for the right 
input of the sound card. The circuit of the sensor 3401 is shown in Figure 
35. Each channel has three microphones connected in parallel. In this 
arrangement, constructive and destructive interference occurs physically 
and electrically. 

In the sensor circuit shown in Figure 35, constructive and destructive 
interference occurs in the electrical domain as a result of electrically mixing 
the signals from transducer means at several locations on the surface. This 
electrical interference is therefore considered as part of the transmission 
system, when a circuit of this type is provided, and may be seen as an 
extension of the complexity of the transmission domain into a different 
medium. Such mixing may also be performed after microphone signals 
have been digitised, resulting in constructive and destructive interference in 
the time domain of digital samples. In a further possible configuration, time 
domain signals may be transformed into another domain, for example by 
the fast fourier transform, and the transformed results combined to form a 
single channel upon which analysis is then performed. 
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Claims 

1. Apparatus for inputting data into a processing system, 
comprising a fricative surface for receiving user finger movements made 
against that surface, and transducer means arranged to receive sounds 
arising from the movement of a user's fingertip across said fricative surface; 
wherein: 

said surface has a direct path and at least one reflection path for 
acoustic conduction of surface sound to said transducer means; such that 
interference between said paths occuring at said transducer means results 
in interference artefacts that characterise the location of said fingertip on 
said surface. 

2. Apparatus according to claim 1, wherein said transducer 
means is located asymettrically with respect to acoustic properties of said 
surface. 

3. Apparatus according to claim 1 or 2, wherein said transducer 
means is a contacting transducer means preferentially responsive to sound 
transmitted through the medium of said surface. 



4. Apparatus according to any of claims 1 to 3; wherein said 
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surface includes acoustic transmission modifying means. 

5. Apparatus according to any of claims 1 to 4, having two said 
transducer means, and further including connecting means for supplying 
signals from said transducer pair to an input of a processing system. 

6. Apparatus according to any of claims 1 to 5, wherein a 
plurality of said transducer means at different locations have outputs that 
are electrically combined. 

7. Apparatus according to any of claims 1 to 6, including digital 
radio transmitting means for transmitting signals from said transducer 
means to a processing means for identifying a location of said surface 
sound. 

8. Apparatus according to any of claims 1 to 7, wherein said 
surface is curved. 

9. Apparatus according to claim 8, wherein said curved surface 
forms a matching portion with another such apparatus, such that the 
combination of a plurality of such apparatus is a fully enclosed curved 
surface. 
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10. Apparatus according to any of claims 1 to 8, wherein said 
surface is part of a portable processing system. 

11. A method of inputting data into a processing system, in which 
signals relating to user input movements on a surface are analysed to 
generate input data for said processing system; wherein 

said signals are generated by transducer means in response to 
sound arising from a user input gesture being made upon said surface; 

said surface sound is transmitted to said transducer means by a 
system of acoustic transmission paths, including a direct path and at least 
one reflection path; and 

the location of the sound on the surface is identified by processing a 
characteristic of said path system with characteristics of other path systems 
of known surface location. 

12. A method according to claim 11, including supplying signals 
from a pair of said transducer means to said processing system. 

13. A method according to claim 1 1 or 12, wherein signals from a 
plurality of said transducer means are analysed independently to identify 
respective preliminary sound source locations. 
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14. A method according to any of claims 11 to 13, wherein said 
paths are modified by acoustic transmission modifying means in said 
surface. 

15. " A method according to any of claims 11 to 14, including 

transforming samples of said transducer signals into location characterising 

j 

data. 

16. A method according to any of claims 11 to 15, including 
electrically combining signals from a plurality of transducers. 



17. A method according to any of claims 11 to 16, including 
transmitting digital representations of signals from said transducer means to 

15 a processing means over a radio link. 

18. A method according to any of claims 11 to 16, including 
supplying said transducer signals to processing means in a portable 
processing system of which said surface is a part. 
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19. A method of processing a signal to determine a location of a 
sound source on a surface, wherein said signal has a characteristic that is 
determined by acoustic reflections in said surface, comprising the steps of: 

characterising said location by transforming said signal to produce 
transformed data that reveals reflection artefacts; and 

identifying a location for said sound source by processing said 
transformed data with a reflection model for said surface. 

20. A method according to claim 19, wherein said reflection model 
is a set of templates. 
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